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A mTE ON lAWLEY'S FORMULAS FOR STANDARD ERRORS IN 
MAXIMUM LIKELIHOOD FACTOR ANALYSIS 

Abstract 

Evidence is given to indicate that Lawley's formulas for the standard 
errors of maximum likelihood loading estimates do not produce exact 
asymptotic results. A small modification is derived which appears to 
eliminate this difficulty. 



A NOTE OK lAWLEY'S FOR STANDARD ERRORS IN 

MAXIMUM LKELIHOOD FACTOR ANALYSIS 

1> Introduction 

In two important papers Lawley [1955, 196?] derived formulas for the 
standard errors of factor loading estimates produced in maximum likelihood 
factor analysis. Recent work [Archer & Jennrich^ 1975; Jennrioh^ 1975a] 
has made it possible to use Lawley' s results to obtain standard errors for 
analytically rotated loadings as well. There is some evidence, however, 
that Lawley' s formulas do not produce exact asymptotic results. We shall 
attempt to demonstrate that this is in fact the case and propose a modifi- 
cation which will make Lawley' s results asjTnptotically exact. This 
modification amounts to the insertion of a single already defined symbol 
into his formulas. While this cliange may produce little practical 
^,ffect in standard error estimates computed from real data, the modifica- 
tion is necessary to eliminate discrepancies which arise when Lawley 's 
formulas are used in conjunction with asymptotic results from other sources. 

The evidence that there is a problem is quite simple. Table 1 contains 
standard errors for maxim.um likelihood loading estimates arising in an 
example given by Lawley and Maxwell [1971, p. 65] and obtained through 
the use of Lawley* s formulas. The authors have recomputed these values 
and obtained results which agree with those of Lawley and Maxwell to within 
one digit in the lasb decimal place presented. This makes us confident 
that both we and they have implemented the required formulas correctly. 
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Insert Table 1 about here 



Table 2 contains asymptotic standard errors for the same example which 
led to Table 1, but which were obtained by inverting the appropriate augmented 
information matrix [jennrich, 1975b]. The differences in these two tables, 
while not large, demonstrate clearly that they cannot both represent true 
asymptotic standard errors. 



Insert Table 2 about here 



Finally, Table 5 contains standard errors for the same example using 
Lawley's formulas with the modification proposed in Section 5. The 
nearly perfect agreement between Tables 2 and 5 is strong evidence that 
with this modification, exact standard errors are obtained through the use 
of Lawley's formulas. 



Insert Table 5 about here 



Because Lawley's formulas are more efficient than those of Jennrich 
[1975b] and avoid the inversion of a large matrix, the authors feel this 
modification is important whenever exact asymptotic standard errors are 
required. 
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2. Locating the Difficulty 

Lawley originally [1955] derived his standard error formulas under the 
assumption that the unique variances ^ in a factor analytic decomposition 

(1) E = AA' + 1]/ 

of a population covariance matrix Z were known. The formulas vere later 
[1967] generalized to the case of unknown unique variances. The difficulty 
lies in this latter article. The results there are derived in greater de- 
tail in Lawley' s text [1971] co-authored with Maxwell. Because of this and 
because the text contains extensions of the original results to cover the 
effect of standardization, we shall base our comments on the 1971 text 
ratner than on the 1967 article. Unless otherwise indicated we shall use 
the notation and definitions of the text. 

On page 55 Lawley and Maxwell define functions g^^(t,S) to be 
the right-hand side of (5-15) in the text divided by (0^ - l) regarded 
as a function of t|/ and a sample covariance matrix S . They assert 
that (5.15) becomes (5.21) which is 

(2) t. - A. = g. (^,S) . 

iro ir ir ^ ' 



iro 



This like (5 '15)^ however, is only an approximate equality. Here A. 
denotes the maximum likelihood estimate of the population loading A 

ir 

computed under the assumption that ^ is known. To computed standard 
errors In the unrestricted case Lawley and Maxwell introduce coefficients 

(5) b ^^=problim|i^l , 

n CO o'^j 



and express them in terms of population parameters by means of equation 
(5.27) of the text which is 



(k) b. . = -X. (0 -l)"^r^[S. X. X../(9 - 1) 

+ Z'A. A. /(e - 9 )] 
m im jm' ^ r m 

Here 0 denotes the r -th largest eigenvalue of i|r'^Zi|r"^ , 6.. denotes 

the Kronecker delta^ and the symbol denotes summation on m from 

m 

1 to the number of factors k skipping the value r . Because of the 
approximation in (2) we cannot agree that (^4)^ when used in (5.29) of the 
text, gives asymp-cotic covariances for the maximum likelihood estimates 
'^^^ of the population loadings A^^ . In order to isolate the problem 
let g^^(^^S) be the function which maps }jf and S into A_.___ minus 



iro 



A^^ . Then replacing g^^ by g^^ makes the appro:cimate equality in 

(2) an exact equality. Similarly let b. . be the value obtained when 

g. is replaced by g in the definition of b . . . As will be seen 

•LI J, ir 

in the next section these modifications lead to a formula for b. . 
which differs slightly from that for b. . . When, however b. . re- 
places b in formula (5-29) of the text the modified standard 

error formulas appear to give exact standard errors as observed in the 
previous section. 
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3> Deriving the Required Modification 



Proceeding here with exact equalities only. 



(5) 'K. - = g. (^^;S) 

iro ir ^ir^"^^ ^ 



and 



ir 



(6) ir = P''^^ 

n -> <» J 



.-Since Z is the probability limit of S ; 



^ir ^ir 

(T) m~ wr 

0 0 



in probability as n ^ • Consequently (6) becomes 



(8) 



5?' 



ir 



3 



What is required are the values of the partial derivatives bg. /bf at 
(\lf,S) = (^,L) . These are a little difficult to obtain, but implicit 
differentiation seems to work nicely. 

Relations (5. 21) and (U.^) from the text and the fact that A'\lf~'^A 
is by assumption (p. 2?) diagonal may be expressed in the form: 



(9) ^0 ' ^ ~' ^^"^'^^ 



(10) 



1^ 



(S - AqA' - ^)^ Aq ^- 0 



(11) non-diag Aq^'^^A^ = 0 
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In order to compute the partial derivatives Sg. at (^,Z) by 

ir J 

implicit differentiation, we differentiate (lO) and (ll) holding S fixed 
at Z and evaluate the result at (^^S^Aq) = {^yZyA) . This leads to the 
differential relations: 

(12) (AdA' + dAA' + d\l/H"'"^A = 0 

(15) non-diag(dA'\l/"'"^A + A^''^dA - A'\lr""^d\lrf -^A) = 0 

Contributions from the second and third factors in (lO) disappear since the 
first term is zero when (it'^S^A^) = (^^L^A) . The required partial deriva- 
tives are found by solving (l2) and (l5) for dA in terms of . This 
may be accomplished by algebraic manipulations similar to those which led 

to (5-15) in the text. 

Lettiro" 

{ik) A A'\1/"'"^A 

which IS a diagonal matrix and multiplying (12) on the left by A'^''^ 
gives 

(15) ^'\lr""^A A'\lr"-^dAA+ A'\l/"'-^dM""^A = 0 

Multiplying (l5) on the right by A and subtracting the result from the non- 
diagonal part of (15) gives 

(16) non-diag(z^dA'*""'A - dA'\lr"'"^AA A'^""^dt^r"^A(l + A)) = 0 



Solving (15) and (16) for the diagonal and nondiagonal components of 
dA'\^ respectively gives 

(17) (dA'^'^A)^^ = -(2A^)"^(A'^"V*"^A)^^ 
and 

(18) (dA'>l/"^A)^g = -(A^ - A^)"^(A'*''W"'^A)^Jl + 

for all r s. Here A^ denotes the r -th diagonal element of A 

and ( • ) denotes the e"*^ment in row r and column s of the 
r s 

expression inside the parentheses. Using (ih), equation (12) can be 
written in the form 

(19) dAA = -dM"''"A - AdA'^~\\ 

Writing this in coordinate form and using (17) and (18) gives 

(20) d-A.^A^ = -(d^'^-A).^ - \„(<i^'* A)_^ 

= -(d<I/M;"^A).^ + \^(2A^y\A'rhn'h)^^ 

Hi ^ A^) z \JA^ - ^^)-\A^rhn-h) . 

mjtr 

The partial derivative ~^\^/'dt. evaluated at (^,L) is the value of 

dA.^ corresponding to d* = j(j,j) where j(j,j) is the p by p matrix 

which has a one in its J -th diagonal position and is zero elsewhere. 

Since b is the value of this partial derivative, replacing dA 

' ir 
by b^ and dif by J(j,j) in (20) gives 
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We will put this in a form which looks as similar as possible to that of 
(5-27) of the text. To that end we note that because of (k.l2) of the text, 

(22) A = e - 1 . 
% / r r 

Making this replacement in (2l) a?. rearranging a bit gives 




The only difference between this and (5* 2?) which is reproduced in (k) is 
the appearance of 0^ immediately preceding the summation symbol. 

k. Modification for Standardization 

The results of the previous sections dealt with standard errors for 
maximum likelihood estimates of (natural) factor loadings. It is common 
in practice to estimate standardized loadings 

(2l^) = X. /a. 

^ ^ ir ir' 1 

Here ix^ denotes the population standard deviation of the i -th score* 
Maximum likelihood estimates of standardized loadings are computed from a 
sample correlation* matrix in precisely the same way that maximum likelihood 
estimates for natural loadings are computea from a sample covariance matriiv. 
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Lawley and Maxwell [1971, 6l] give standard erro 
standardized loading estimates in addition to those air 
the unstandardized case. Since these formialas also invo: 
derivatives expect that they vill not give exi 

results and that they could be made exact if the b 
• This in fact seems tc be the case* 

0,ir 

Lawley and Maxwell's formulas for standardized load! 
two alternative forms. The first, which is based on (5.1^ 
^^^^ix ^^li^i^^ requires no modification excef^t for 
of ^i,iT • "^^^ second is based on (5*^7) and 

quired modification involves replacing the -1 in the firs 

the suim'ttion symbol by -0 . The modified formulas in 

r 

the text are 

(25) n cov(A s ) . 2 Z (A. a ) + 2t% . . 
and 

(26) n cov(A s ) = (0 . 1)'^^ f29 a.. - 26..^. - 

-^^ -iJ - r ij ij 1 

+ 2 £'(9 9 - © ){9 - e 

r s r^^ r s^ 

s 

To check these moaifications the authors have impleme 
sions of lawley and Maxv.-ell's fomulas and applied them to 
discussed in Section 1. Both versions produced results wh 
within one digit in the Last decimal place with those give 
Maxwell [I97I, p. 61*]. The latter are reproduced in Table 
^ ment suggests again that all parties correctly iraplementpd 

ERIC 



-10- 



the text. Table 5 contains the corresponding asymptotic standard errors ob 
taiced by inverting the appropriate augmented information matrix [Jennrich 
1975b]. It displays the anticipated lack of agreement with the results in 
Table k. Finally^ the results in Table 6 were obtained using the first 
form of the Lawley and Maxwell formulas with the modification discussed in 
this section. The agreement between Tables 5 and 6, which again is to 



Insert Tables Uy 5^ and 6 about here 

within one digit in the last decimal place presented, suggests that the 
proposed modification does produce exact asymptotic standard errors. Using 
the second version of the Lawley and MaxT^/ell formulas v;ith the proposed 
modification produced no surprises. The results were identical to those in 
Table 6. 

^. Comments 

We have avoided the assertion that Lav/ley's standard error results 
are incorrect. To our knowledge it v;as never claimed that they were exact 
asymptotic results. V/hat is important here we believe is that v/e have 
furnished an almost completely independent verification of a slightly modi- 
fied form of his formulas. For this the modification played a necessary 
but not the central role. At present, the whole area of standard errors 
for factor loading estimates is fairly new and fairly complicated. Since 
there is little that is as fallible as a mathematical derivation, 
independent verifications like those presented here are not only com- 
forting but essential. 
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TABLE 1 

Standard Errors Using Lav;ley's Formulas 



Variate 


I 


Factor 
II 


III 


1 


.066 


.058 


.076 


2 


.06h 


.061 


.068 


5 


.070 


.071 


.085 


l» 


.060 


.ohe 


.Ol<5 


5 


.065 


.057 


.072 


6 


.06i< 


.Ohe 


.057 


7 


.068 


.057 


.066 


8 


.073 


.095 


.11)2 


9 


.066 


.OU7 


.055 
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TABLE 2 

Standard Errors Using an Augmented Information Matrix 



Variate 


I 


Factor 
II 


III 


1 


.068 


.066 


.091 


2 


.066 


.076 


.069 


5 


.072 


.081 


.081* 


k 


.061 


.068 


.05i< 


5 


.067 


.075 


.081 


6 


.069 


,0k9 


.056 


7 


.071 


.068 


.080 


8 


.076 


.105 


.12h 


9 


.070 


.058 


.078 
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TABLE 5 

Standard Errors Using the Modified Lawley Formulas 



Variate 


I 


Factor 
II 


III 


1 


.068 


.066 


.091 


2 


.066 


.076 


.069 


5 


.072 


.081 


.081* 


li 


.061 


.067 


.O^h 


5 


.067 


•075 


.081 


6 


.069 


.0l<9 


.056 


7 


.071 


.068 


.080 


8 


.076 


.105 


.12k 


9 


.070 


.058 


.078 
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TABLE h 

Standard Errors for Standardized Loading 
Using Lawley and Maxwell's Formulas 



Variate 


I 


Factor 
II 


III 


1 


.Okh 


.057 


.076 


2 


.Ohl 


.060 


.068 


5 


.058 


.069 


.082 


k 


.027 


.0k8 


.01)5 


5 


.OkO 


.057 


.073 


6 


.052 


.050 


.058 


7 


.Oh6 


.056 


.066 


8 


.062 


.090 


.IJjO 


9 


.057 


.050 


.055 
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TABLE 5 

Standard Errors for Standardized Loadings Using an 
Augmented Information Matrix 



Variate 


I 


Factor 
II 


III 


1 




.065 


.091 


2 




.076 


.069 


5 


.060 


.078 


.085 


1) 


.050 


.069 


.05^ 


5 


.Oi»5 


.075 


.081 


6 


.0Jj2 


.052 


.057 


7 


.050 


.067 


.080 


8 


.066 


.101 


.121 


9 




.060 


.078 



ERIC 
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TABLE 6 

Standard Errors for Standardized Loadings Using the 
Modified Lawley and Maxwell Formulas 



Variate 


I 


Factor 
II 


III 


1 




.065 


.091 


2 




.076 


.069 


5 


.060 


.078 


.085 


1) 


.050 


.069 




5 


.Ok3 


.075 


.081 


6 


.0k2 


.052 


.057 


7 


.050 


.067 


.080 


8 


.066 


.101 


.122 


9 


• OJjJj 


.060 


.078 



